Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Remote Store] Update Translog Metadata file name #8350

Merged
merged 6 commits into from
Jul 7, 2023

Conversation

gbbafna
Copy link
Collaborator

@gbbafna gbbafna commented Jun 29, 2023

Description

We are changing translog metadata file name to

<inverted_primary_term>-- <inverted_generation>--<inverted_timestamp>--

This would enable us to use lexicographic sort given by remote stores and find the latest metadata in just one list call, instead of doing pagination .

With this change, we are also deleting the older metadata files with every cleanup operation.

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@gbbafna gbbafna force-pushed the new-metadata-format branch from 1db2b68 to 0c29a67 Compare June 29, 2023 13:53
@gbbafna gbbafna changed the title [Remote Store] New metadata file name [Remote Store] Update Translog Metadata file name Jun 29, 2023
@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@gbbafna gbbafna force-pushed the new-metadata-format branch from 0c29a67 to 3931db0 Compare July 3, 2023 03:27
@github-actions
Copy link
Contributor

github-actions bot commented Jul 3, 2023

Gradle Check (Jenkins) Run Completed with:

@gbbafna gbbafna force-pushed the new-metadata-format branch from 3931db0 to 7acb962 Compare July 3, 2023 03:55
@github-actions
Copy link
Contributor

github-actions bot commented Jul 3, 2023

Gradle Check (Jenkins) Run Completed with:

@gbbafna gbbafna force-pushed the new-metadata-format branch 2 times, most recently from 1d519b7 to 5c5c85f Compare July 4, 2023 05:17
@github-actions
Copy link
Contributor

github-actions bot commented Jul 4, 2023

Gradle Check (Jenkins) Run Completed with:

@gbbafna gbbafna force-pushed the new-metadata-format branch 2 times, most recently from e512bb1 to 6991096 Compare July 7, 2023 13:33
@github-actions
Copy link
Contributor

github-actions bot commented Jul 7, 2023

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

github-actions bot commented Jul 7, 2023

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

github-actions bot commented Jul 7, 2023

Gradle Check (Jenkins) Run Completed with:

gbbafna added 6 commits July 7, 2023 22:09
Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
@sachinpkale sachinpkale force-pushed the new-metadata-format branch 2 times, most recently from 194262a to c80a390 Compare July 7, 2023 16:45
@github-actions
Copy link
Contributor

github-actions bot commented Jul 7, 2023

Gradle Check (Jenkins) Run Completed with:

  • RESULT: UNSTABLE ❕
  • TEST FAILURES:
      1 org.opensearch.index.shard.RemoteStoreRefreshListenerTests.testRefreshSuccessOnSecondAttempt

@sachinpkale sachinpkale merged commit ed38fa2 into opensearch-project:main Jul 7, 2023
@sachinpkale sachinpkale added the backport 2.x Backport to 2.x branch label Jul 7, 2023
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-8350-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 ed38fa28a96e111bf12e04c67847c2396151fe19
# Push it to GitHub
git push --set-upstream origin backport/backport-8350-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-8350-to-2.x.

@github-actions
Copy link
Contributor

github-actions bot commented Jul 7, 2023

Gradle Check (Jenkins) Run Completed with:

@ashking94 ashking94 deleted the new-metadata-format branch July 8, 2023 03:15
gbbafna added a commit to gbbafna/OpenSearch that referenced this pull request Jul 8, 2023
…#8350)

---------

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
sachinpkale pushed a commit that referenced this pull request Jul 8, 2023
---------

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
vikasvb90 pushed a commit to raghuvanshraj/OpenSearch that referenced this pull request Jul 12, 2023
…#8350)

---------

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
raghuvanshraj pushed a commit to raghuvanshraj/OpenSearch that referenced this pull request Jul 12, 2023
…#8350)

---------

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
dzane17 pushed a commit to dzane17/OpenSearch that referenced this pull request Jul 12, 2023
…#8350)

---------

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
buddharajusahil pushed a commit to buddharajusahil/OpenSearch that referenced this pull request Jul 18, 2023
…#8350)

---------

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
Signed-off-by: sahil buddharaju <sahilbud@amazon.com>
baba-devv pushed a commit to baba-devv/OpenSearch that referenced this pull request Jul 29, 2023
…#8350)

---------

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
shiv0408 pushed a commit to Gaurav614/OpenSearch that referenced this pull request Apr 25, 2024
…#8350)

---------

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch skip-changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants